1. Field of the Invention
The present invention relates to an image interpolating method for interpolating an image.
2. Description of the Prior Art
Conventionally in cases such as a case where an interlace image is converted into a progressive image, a case where an image is enlarged, and a case where the resolution of an image is enhanced, various methods for interpolating an image have been proposed using pixel data on adjacent original pixels (corresponding to the amount of data representing luminance on a display).
Typical examples of an image interpolating method are a simple interpolating method and a linear interpolating method. The simple interpolating method is a method of giving pixel data on either one of pixels adjacent to a pixel to be interpolated on the upper and lower sides (or on the right and left sides) to the pixel to be interpolated as pixel data. The linear interpolating method is a method of giving the average of pixel data on pixels adjacent to a pixel to be interpolated on the upper and lower sides (or on the right and left sides) to the pixel to be interpolated as pixel data.
In the simple interpolating method, however, the pixel data on the pixel adjacent to the pixel to be interpolated is given as it is. When there is a diagonal edge portion in an image reproduced on a display or the like, therefore, backlash occurs in the edge portion. On the other hand, in the linear interpolating method, the average of the pixel data on the pixels adjacent to the pixel to be interpolated on the upper and lower sides (on the right and left sides) is given. Accordingly, the vicinity of the pixel to be interpolated is an edge portion. When the difference between the pixel data on the pixels adjacent to the pixel to be interpolated on the upper and lower sides (on the right and left sides) is large, the interpolated pixel takes an intermediate value, so that the edge portion is blurred.
Therefore, the applicants of the present invention already have developed an image interpolating method capable of preventing, in interpolating an image, backlash and blur from occurring in an edge portion of the image as well as capable of reproducing a smooth image.
Description is made of an image interpolating method already developed by the applicants of the present invention (hereinafter referred to as the conventional method).
A two-dimensional image has a two-dimensional expanse in the horizontal direction and the vertical direction. In order to simplify the description, however, description is made of a method of one-dimensional interpolation in the vertical direction.
FIG. 1 illustrates the relationship between original pixels and a pixel to be interpolated.
In FIG. 1, lines (n−1), n, (n+1), and (n+2) are horizontal lines in the original image, and lines (i−1), i, and (i+1) are horizontal lines to be interpolated. Description is herein made of a case where a pixel (hereinafter referred to as an interpolation pixel X) on the line i is interpolated.
In the following description, pixel data on original pixels D01 to D35 and the interpolation pixel X are respectively denoted by d01 to d35 and x. In this example, the pixel data shall be composed of eight bits, and black data and white data shall be respectively “0” and “255”.
FIG. 2 shows the procedure for finding the pixel data x on the interpolation pixel X by the conventional method.
First, an edge component E is calculated in order to judge whether or not the interpolation pixel X exists in the vicinity of an edge of an original image (step 1). That is, the edge component E is found from the following equation (1) using the pixel data on the two original pixels D01 and D13 just above the interpolation pixel X and the two original pixels D23 and D33 just below the interpolation pixel X:E=−d03+d13+d23−d33  (1)
The edge component E takes a relatively large negative value when the interpolation pixel X exists in the vicinity of a black edge of the original image, while taking a relatively large positive value when the interpolation pixel X exists in the vicinity of a white edge of the original image.
A range S where the pixel data x on the interpolation pixel X is settable is then determined on the basis of the edge component E (step 2).
Description is made of a method of determining the settable range S on the basis of FIGS. 3a and 3b. In FIGS. 3a and 3b, dmax denotes the larger one of the pixel data d13 and d23 on the original pixels D13 and D23, and dmin denotes the smaller one of the pixel data d13 and d23. Further, dc denotes the average (dmax+dmin)/2 of dmax and dmin.
(1) Range S Where Pixel Data x is Settable When E≧0
When E≧0, the range S where the pixel data x is settable shall be a range of dmin×α+dc×(1−α)≦S≦dmax×α+dc×(1−α)+E×γ, where α and γ are variables which can be controlled from the exterior.
That is, when the edge component E is not less than zero, the range S where the pixel data x is settable is a range expanded upward by the value of γ·E from a range [dmin×α+dc×(1−α)≦x≦dmax×α+dc×(1−α)] centered around dc.
In this case, it is presumed that the interpolation pixel X exists in the vicinity of the white edge of the original image, to set the settable range S to a range close to dmax in the range of dmin to dmax. 
(2) Range S Where Pixel Data x is Settable When E<0
When E<0, the range S where the pixel data x is settable shall be a range of dmin×α+dc×(1−α)+E×γ≦S≦dmax×α+dc×(1−α).
That is, when the edge component E is less than zero, the range S where the pixel data x is settable is a range expanded downward by the value of γ·E from a range [dmin×α+dc×(1−α)≦x≦dmax×α+dc×(1−α)] centered around dc.
In this case, it is presumed that the interpolation pixel X exists in the vicinity of the black edge of the original image, to set the settable range S to a range close to dmin in the range of dmin to dmax.
Candidates for the pixel data x on the interpolation pixel X are then found from the range S where the pixel data x is settable on the basis of the pixel data on the opposed pixels between which the interpolation pixel X is sandwiched diagonally (step 3).
In finding the candidates for the pixel data x on the interpolation pixel X, a correlation value L1 between the pixel data on the opposed pixels D12 and D24 and the pixel data x on the interpolation pixel X and a correlation value R1 between the pixel data on the opposed pixels D14 and D22 and the pixel data x on the interpolation pixel X are employed. Therefore, methods of respectively calculating the correlation values L1 and R1 will be first described.
The correlation value L1 between the pixel data on the opposed pixels D12 and D24 and the pixel data x on the interpolation pixel X is expressed by the following equation (2), and the correlation value R1 between the pixel data on the opposed pixels D14 and D22 and the pixel data x on the interpolation pixel X is expressed by the following equation (3):L=|d12−x|+|d24−x|+β1×Hl−β2×Vl  (2)R=|d14−x|+|d22−x|+β1×Hr−β2×Vr  (3)
In the foregoing equations (2) and (3), β1 and β2 are valuables which can be controlled from the exterior. In the foregoing equation (2), Hl denotes a horizontal correlation intensity, and is expressed by the following equation (4). Further, Vl denotes a vertical correlation intensity, and is expressed by the following equation (5).Hl=MAX(|d11−d12|+|d12−d13|, |d23−d24|+|d24−d25|)  (4)Vl=MIN(|d02−d12|+|d12−d22|, |d14−d24|+|d24−d34|)  (5)
MAX (a, b) is a sign indicating that the larger one of a and b is selected. MIN (a, b) is a sign indicating that the smaller one of a and b is selected.
Hl denotes the larger one of a primary differential value (|d11−d12|+|d12−d13|) in the horizontal direction corresponding to the pixel D12 and a primary differential value (|d23−d24|+|d24−d25|) in the horizontal direction corresponding to the pixel D24. Further, Vl denotes the smaller one of a primary differential value (|d02−d12|+|d12−d22|) in the vertical direction corresponding to the pixel D12 and a primary differential value (|d14−d24|+|d24−d34|) in the vertical direction corresponding to the pixel D24.
In the foregoing equation (3), Hr denotes a horizontal correlation intensity, and is expressed by the following equation (6), and Vr denotes a vertical correlation intensity, and is expressed by the following equation (7):Hr=MAX(|d13−d14|+|d14−d15|, |d21−d22|+|d22−d23|)  (6)Vr=MIN(|d04−d14|+|d14−d24|, |d12−d22|+|d22−d32|)  (7)
Hr denotes the larger one of a primary differential value (|d13−d14|+|d14−d15|) in the horizontal direction corresponding to the pixel D14 and a primary differential value (|d21−d22|+|d22−d23|) in the horizontal direction corresponding to the pixel D22. Further, Vr denotes the smaller one of a primary differential value (|d12−d22|+|d22−d32|) in the vertical direction corresponding to the pixel D22 and a primary differential value (|d04−d14|+51 d14−d24|) in the vertical direction corresponding to the pixel D14.
There are two methods of finding the candidates for the pixel data x on the interpolation pixel X. The methods will be described.
(1) Description of First Method
All the pixel data x in the settable range S are respectively substituted in the foregoing equation (2) expressing a correlation value L between the pixel data on the two opposed pixels D12 and D24 between which the interpolation pixel X is sandwiched diagonally (diagonally to the upper left) and the pixel data x on the interpolation pixel X, to find the minimum correlation value Lmin and pixel data xl in a case where the minimum correlation value Lmin is given.
Similarly, all the pixel data x in the settable range S are respectively substituted in the foregoing equation (3) expressing a correlation value R between the pixel data on the two opposed pixels D14 and D22 between which the interpolation pixel X is sandwiched diagonally (diagonally to the upper right) and the pixel data x on the interpolation pixel X, to find the minimum correlation value Rmin and pixel data xr in a case where the minimum correlation value Rmin is given.
The found pixel data xl and xr are thus candidates for the pixel data x on the interpolation pixel X. In such a way, however, a plurality of pixel data may, in some cases, be applicable as the pixel data xl in the case where the minimum correlation value Lmin is given. Similarly, a plurality of pixel data may, in some cases, be applicable as the pixel data xr in the case where the minimum correlation value Rmin is given.
When the plurality of pixel data are applicable as the pixel data xl in the case where the minimum correlation value Lmin is given, the average of the pixel data may be determined as the pixel data xl in the case where the minimum correlation value Lmin is given. Similarly, when the plurality of pixel data are applicable as the pixel data xr in the case where the minimum correlation value Rmin is given, the average of the pixel data may be determined as the pixel data xr in the case where the minimum correlation value Rmin is given.
(2) Description of Second Method
FIG. 4 illustrates the relationship between the correlation value L and the pixel data x. In FIG. 4, xmin denotes the smaller one of the pixel data d12 and d24, and xmax denotes the larger one of the pixel data d12 and d24. Further, xd=xmax−xmin+β1×Hl−β2×Vl.
If xd=xmax−xmin+β1×Hl−β2×Vl, the correlation value L in the foregoing equation (2) can be changed, as expressed by the following equation (8). The correlation value R can be also similarly changed.
                                                        L              =                              xd                +                                  2                  ⁢                                      (                                          x                      -                                              x                        max                                                              )                                                                                                          (                              x                >                                  x                  max                                            )                                                                          L              =              xd                                                          (                                                x                  min                                ≦                x                ≦                                  x                                      max                    ⁢                                                                                                                            )                                                                          L              =                              xd                +                                  2                  ⁢                                      (                                                                  x                        min                                            -                      x                                        )                                                                                                          (                              x                <                                  x                  min                                            )                                                          (        8        )            
Description is now made of methods of respectively finding the minimum correlation values Lmin and Rmin and the pixel data xl and xr (the second method). Description is herein made of the method of finding the minimum correlation value Lmin and the pixel data xl.
FIGS. 5a to 5f show graphs of the correlation value L against the pixel data x and a range where the pixel data x is settable.
In FIGS. 5a to 5f, xa denotes the minimum of the pixel data x in the settable range, and xb denotes the maximum of the pixel data x in the settable range.
{circle around (1)} In the case of xb≦xmin 
When the range where the pixel data x is settable is not more than xmin, as shown in FIG. 5a, the correlation value L is the minimum in the pixel data xb. Consequently, the minimum correlation value Lmin is xd+2(xmin−xb). Further, the pixel data xl at this time is xb.
{circle around (2)} In the case of xmax≦xa
When the range where the pixel data x is settable is not less than xmin, as shown in FIG. 5b, the correlation value L is the minimum in the pixel data xa. Consequently, the minimum correlation value Lmin is xd+2 (xa−xmax). Further, the pixel data xl at this time is xa.
{circle around (3)} In the case of xmin≦xa and xb≦xmax 
When the range where the pixel data x is settable is from the pixel data xmin to xmax, as shown in FIG. 5c, the correlation value L is the minimum xd in the arbitrary pixel data x in the settable range. Consequently, the minimum correlation value Lmin is xd. In this case, a value (xa+xb)/2 at the center of the settable range is determined as the pixel data xl.
{circle around (4)} In the case of xa<xmin and xmax<xb
When the range where the pixel data x is settable is a range wider than the range of the pixel data xmin to xmax, as shown in FIG. 5d, the correlation value L is the minimum xd in the arbitrary pixel data x in the range of the pixel data xmin to xmax. Consequently, the minimum correlation value Lmin is xd. In this case, the average (xmin+xmax)/2 of the pixel data xmin and xmax is determined as the pixel data xl.
{circle around (5)} In the case of xa<xmin and xmin<xb≦xmax 
When the range where the pixel data x is settable is shifted leftward from the range of the pixel data xmin to xmax, as shown in FIG. 5e, the correlation value L is the minimum xd in the arbitrary pixel data x in the range of the pixel data xmin to xb. Consequently, the minimum correlation value Lmin is xd. In this case, the average (xmin+xb)/2 of the pixel data xmin and xb is determined as the pixel data xl.
{circle around (6)} In the case of xmin≦xa<xmax and xmax≦xb
When the range where the pixel data x is settable is shifted rightward from the range of the pixel data xmin to xmax, as shown in FIG. 5f, the correlation value L is the minimum xd in the arbitrary pixel data x in the range of the pixel data xa to xmax. Consequently, the minimum correlation value Lmin is xd. In this case, the average (xa+xmax)/2 of the pixel data xa and xmax is determined as the pixel data xl.
The minimum correlation value Lmin and the pixel data xl are thus found. The minimum correlation value Rmin and the pixel data xr are also found in the same manner as the minimum correlation value Lmin and the pixel data xl.
When the minimum correlation values Lmin and Rmin and the pixel data xl and xr in cases where the minimum correlation values are respectively given are found in the step 3, the pixel data in the case where the smaller one of the minimum correlation values Lmin and Rmin is given is extracted (step 4).
When the minimum correlation values Lmin and Rmin differ from each other, one pixel data is extracted. When the minimum correlation values Lmin and Rmin are the same, two pixel data are extracted.
When one pixel data xl or xr is extracted in the step 4 (YES in step 5), the extracted pixel data is determined as the pixel data x on the interpolation pixel X (step 6). When two (a plurality of) pixel data xl and xr are selected in the step 4 (NO in step 5), the average of the pixel data xl and xr is determined as the pixel data x on the interpolation pixel X (step 7).
The details are as follows:
{circle around (1)} if Lmin<Rmin, then x=xl
{circle around (2)} if Lmin>Rmin, then x=xr
{circle around (3)} if Lmin=Rmin, then x=(xl+xr)/2
Although in the foregoing step 3, two sets are selected as the set of opposed pixels between which the interpolation pixel X is sandwiched diagonally, two or more sets may be selected. For example, six sets of opposed pixels, for example, D10 and D26, D11 and D25, D12 and D24, D14 and D22, D15 and D21, and D16 and D20 may be selected, as shown in FIG. 6. In this case, pixel data in a case where the minimum correlation value is given is found in the step 3 for each of the sets of opposed pixels. That is, six candidates for the pixel data are found.
When the number of minimums of the minimum correlation values corresponding to the six sets of opposed pixels is three or more, three or more pixel data are extracted from the six candidates for the pixel data in the step 4.
When three or more pixel data are selected in the step 4, there are three methods, as described below, as a method of determining the pixel data x on the interpolation pixel X.
The first method is one for calculating the average of three or more pixel data selected in the step 4 and determining the result of the calculation as the pixel data x on the interpolation pixel X. The second method is one for extracting the maximum and the minimum of three or more pixel data selected in the step 4, calculating the average of the maximum and the minimum, and determining the result of the calculation as the pixel data x on the interpolation pixel X.
The third method is one for selecting, out of three or more pixel data selected in the step 4, the pixel data obtained from the opposed pixels in closest proximity to the interpolation pixel X, and determining the selected pixel data as the pixel data x on the interpolation pixel X. When there exist two pixel data obtained from the opposed pixels in closest proximity to the interpolation pixel X, however, the average of the pixel data is calculated, and the result of the calculation is determined as the pixel data x on the interpolation pixel X.
When six sets of opposed pixels are selected, as shown in FIG. 6, as the set of opposed pixels between which the interpolation pixel X is sandwiched diagonally, the correlation values L2 and R2 are respectively expressed by the following equations (9) and (10), for example:L2=|d11−x|+|d25−x|+β1×Hl2−β2×Vl2  (9)R2=|d15−x|+|d21−x|+β1×Hr2−β2×Vr2  (10)
Hl2 and Vl2 in the foregoing equation (9) are respectively expressed by the following equations (11) and (12):Hl2=MAX(|d10−d11|+|d11−d12|, |d24−d25|+|d25−d26|)  (11)Vl2=MIN(|d01−d11|+|d11−d21|, |d15−d25|+|d25−d35|)  (12)
Hr2 and Vr2 in the foregoing equation (10) are respectively expressed by the following equations (13) and (14):Hr2=MAX(|d14−d15|+|d15−d16|, |d20−d21|+|d21−d22|)  (13)Vr2=MIN(|d05−d15|+|d15−d25|, |d11−d21|+|d21−d31|)  (14)